Skip to content

fix: preserve unread input in local streaming writes - #295

Open
perfloop[bot] wants to merge 3 commits into
mainfrom
perfloop-pr-open-phpwnxk5zq
Open

perfloop[bot] wants to merge 3 commits into
mainfrom
perfloop-pr-open-phpwnxk5zq

Conversation

@perfloop

@perfloop perfloop Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Local PutStream writes should consume only the bytes a caller has not already read. The local first attempt starts at the reader's call-entry position, and retries return to that same position. A positioned reader that relied on the old local rewind will now store only its unread suffix. For active calls whose starting position cannot be obtained, storage gets one attempt; a failed retry seek returns an error instead of writing a different slice. No public signature or wire format changes, and remote forwarding is unchanged. The embedded API comment and guide document the local behavior.

TestPutStreamReaderPosition covers positioned bytes.Reader and file inputs, zero and end positions, reader-only and pipe-backed readers, cancellation, and a Seeker whose position query blocks. TestPutLocalReaderRetryPosition injects a retryable failure after a partial read and checks replay. TestPutLocalReaderWithoutPositionAttemptsOnce and TestPutLocalReaderRetrySeekFailure cover unavailable positions and failed restoration. The focused server Put suite exercises the unchanged unary bytes-reader and io.PipeReader callers.

This condition holds with this change and does not hold at 0759cbea25127e1e69a2ac0bb5a25bc429d32ab8: With a live context and a local embedded.Client.PutStream call, a successful write stores exactly the bytes from the reader's call-entry position through EOF; retryable storage failures replay from that offset when it is available, while readers with no obtainable starting position receive one storage attempt from the current position. A failed retry seek returns an error without committing another slice, and an already-canceled seekable call returns a timeout before invoking the reader's Seek method, without consuming or storing the input..

A standalone executable also drives embedded.Client.PutStream through a retryable metadata-commit failure after the body is consumed. The executable and its fault-injection helper are not included in the PR. The shipped operations test covers the separate partial-read retry case. The focused embedded and server Put tests passed, including the race-filtered Put path. TestPut_MetadataCommitFailureReclaimsRawFile passed; it verifies that a raw file is reclaimed after a failed metadata commit. The server build, lint, and license check also passed.

Caller incidence and storage or copying cost remain unmeasured.


Generated by Perfloop.


Note

Medium Risk
Changes core local ingest semantics for partially consumed readers and retry paths; behavior shift for callers that previously relied on full-stream rewind on retry.

Overview
Local streaming puts no longer rewind seekable readers to offset zero. PutStream / PutLocal now store bytes from the reader’s call-entry position through EOF. The first write reads forward from there; on retryable storage failures, putLocalWithRetry restores that captured offset via io.Seeker instead of seeking to the start.

Readers without a reliable position get one storage attempt. A failed retry reposition returns an error (and is not treated as retryable storage). Canceled contexts return a timeout before querying position on seekable inputs, without consuming the body or creating a key.

Docs and Client.PutStream comments describe this local behavior. New embedded integration tests and put_local_retry_test cover positioned inputs, pipes, cancellation, and retry/seek edge cases. Remote gRPC forwarding is unchanged.

Reviewed by Cursor Bugbot for commit c9fed33. Bugbot is set up for automated code reviews on this repo. Configure here.

Signed-off-by: Perfloop Agent <agent@perfloop.ai>
Signed-off-by: Perfloop Agent <agent@perfloop.ai>
Signed-off-by: Perfloop Agent <agent@perfloop.ai>
@greptile-apps

greptile-apps Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] This PR appears safe to merge.

Summary

Local streaming writes preserve the reader’s starting position.

  • Local streaming writes store bytes from the reader’s current position.

perfloop[bot] explicitly acknowledges that positioned readers now store only their unread suffix. Remote forwarding is unchanged. No actionable issues were found.

Reviews (1) · Last reviewed commit: "fix: check cancellation before reader po..." · Reviewed by Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant